VLC technique for layered video coding using distinct element grouping

ABSTRACT

The layered coding technique is employed to achieve the image quality scalability for video coding standards. The desired image quality scalability can be achieved by refining the image coefficients in subsequent enhancement layers. In most cases, the refinement coefficient consists of some binary information such as whether this coefficient is refined in this coding pass, whether this coefficient is positively or negatively refined, etc. Because it is generally difficult to code binary information efficiently with VLC (Variable Length Coding) technology, this disclosure introduces a method to code refinement symbol more efficiently with VLC by grouping the symbols of distinct binary elements.

CLAIM OF PRIORITY

This application claims priority under 35 U.S.C. 119(e)(1) to U.S.Provisional Application No. 60/780,192 filed Mar. 8, 2006.

TECHNICAL FIELD OF THE INVENTION

The technical field of this invention is improving the compressionefficiency with a quality scalability feature, which is applicable tovideo coding and other coding technologies such as imaging, audio, etc.

BACKGROUND OF THE INVENTION

Imaging and video coding standards (such as JPEG, MPEG, and H.26x) haveevolved greatly in the past decade. In the early stages, the mainrequirements and objective for such video coding standards werecompression efficiency. With the expansion of markets, video codingstandards began to require other attractive features in addition to highcompression efficiency. One of the most desired features is imagequality scalability. In most cases, image quality scalability isachieved by a layered video coding technique. This includes a base layerand subsequent separate enhancement layers. The base layer is theminimum of coded data sufficient to generate a visible video sequence.An existing video coding standard can be the base layer. Eachenhancement layer contains data to improve the quality of the decodedimage over the base layer and preceding enhancement layers. In general,these enhancement layers refine the values of transformed coefficients.For example, suppose the desired transformed coefficient to be coded is25. A scalable formation of 25 can be achieved by coding 16 in the baselayer, 8 in the first enhancement layer and 1 in the second enhancementlayer.

There are two main objectives in processing an enhancement layer:propagation of significance; and refinement of magnitude. In propagationof significance, the enhancement layer coder seeks a non-significantcoefficient. Such a non-significant coefficient is not yet coded inpreceding layers. The coder codes the location of the coefficient andits magnitude if necessary. Because the significance of the coefficientwill greatly depend on the neighboring coefficients and the location ofthe coefficient can be encoded in various ways, it is relatively easy toefficiently code the propagation of significance in variable lengthcoding (VLC).

In the refinement of magnitude, the enhancement layer coder seeks asignificant coefficient which is already coded in preceding layers.Unlike the propagation of significance, the refinement of magnitude isalmost independent of the status of neighboring coefficients or thecoefficient itself. In most cases, the coder will generate a binary flagindicating whether this coefficient will be refined or not. This iscalled a refinement flag. Depending on the method of refinement, thesign of refinement (called refinement sign) and the magnitude ofrefinement (called refinement magnitude especially if greater than 1)are also coded. Many binary symbols need to be coded in this refinementof magnitude process. Efficient coding of such binary symbols isimportant to the quality scalability with a desired compressionefficiency.

A VLC technique is generally employed to improve the coding efficiency.A shorter code is assigned to more frequent symbols, and a longer codeto less frequent symbols. Thus a 1 bit codeword will be assigned to asymbol even if that symbol appears with extremely high probability (e.g.nearly 100%). Therefore, it is believed variable length coding of binarysymbols is inefficient. The JPEG 2000 standard employs arithmetic codingtechnology to address this problem. The JPEG 2000 standard is a known asstill image coding standard with quality and other various scalabilityfeatures. However, it is known that arithmetic coding technology is muchmore complex computationally than VLC technology. Therefore, some VLCbased technique that would efficiently code the refinement symbols isdesired.

FIG. 1 illustrates flowchart of a typical refinement symbol codingtechnique 100 using VLC. Technique 100 begins with start block 101.Block 102 scans the coefficients and identifies those to be coded usingthis process. Block 103 divides each refinement coefficient into severalinformational elements. As noted above these informational elementstypically include refinement flag, refinement sign and refinementmagnitude. Block 104 groups the current binary informational elementsymbols. For the first iteration, the current binary informationalelement symbols is the first binary informational element symbols. Block105 codes the grouped symbols using a known VLC technique. Block 106codes any remaining fractional symbols. Test block 107 determines ifthere are remaining binary informational symbols. If there are remainingbinary informational symbols (Yes at test block 107), then block 108starts the next binary informational symbols. Technique 100 returns toblock 104 to start coding this next binary informational symbol. Ifthere are no remaining binary informational symbols (No at test block107), then block 109 uses a VLC technique to code any remainingnon-binary elements. Technique 100 ends at end block 110.

FIG. 2 illustrates an example of the conventional VLC method to coderefinement symbols. This example assumes that a refinement flag, arefinement sign and a refinement magnitude are to be coded. The firstcolumn of the first half is a coefficient index from 0 to 6. The secondcolumn of the first half is the coefficient data already coded inpreceding layers. The third column of the first half is the refinementcoefficients to be coded in the current layer. After coding the currentlayer, the refined coefficients data will be the summation of the secondcolumn and the third column. The second half shows the coding of theserefinement coefficients. The refinement coefficients are divided intothe following three elements: refinement flag (fourth column, binary),refinement sign (fifth column, binary), and refinement magnitude (sixthcolumn, multinary).

For coefficient index 0 the refinement coefficient of the current layeris 0. This is coded using a 0 refinement flag. No refinement sign orrefinement magnitude are coded in this case.

For coefficient index 1 the refinement coefficient of the current layeris 2. This is coded using a 1 refinement flag, a 0 refinement signindicating a positive sign and a refinement magnitude of 2.

For coefficient index 2 the refinement coefficient of the current layeris 1. This is coded using a 1 refinement flag, a 0 refinement signindicating a positive sign and a refinement magnitude of 1.

For coefficient index 3 the refinement coefficient of the current layeris also 1. This is coded using a 1 refinement flag, a 0 refinement signindicating a positive sign and a refinement magnitude of 1.

For coefficient index 4 the refinement coefficient of the current layeris −4. This is coded using a 1 refinement flag, a 1 refinement signindicating a negative sign and a refinement magnitude of 4.

For coefficient index 5 the refinement coefficient of the current layeris 0. This is coded using a 0 refinement flag. No refinement sign orrefinement magnitude is coded.

For coefficient index 6 the refinement coefficient of the current layeris 1. This is coded using a 1 refinement flag, a 0 refinement signindicating positive sign and a refinement magnitude of 1.

Depending on the implementation of this refinement symbol coding, therefinement magnitude may be omitted by restricting the magnitude to 1.The refinement sign may be omitted by allowing only positive refinementcoefficients. Those up to these three elements are coded in this VLCtechnique.

As previously noted, VLC coding of binary symbols tends to beinefficient. Each symbol requires one bit regardless of the probabilityof the coded symbols. A grouping approach may be employed in someimplementations to improve compression efficiency. The grouping approachbundles several binary symbols of the same element into one new symbol.Symbols of the same element are expected to have almost equivalentprobability of 0 or 1. A codeword is assigned to each possible bundlednew symbol.

FIG. 3 illustrates an example of such a grouping of length 2. Groupedsymbols 00 are coded as 0. Grouped symbols 01 are coded as 10. Groupedsymbols 10 are coded as 110. Grouped symbols 11 are coded as 111. In theexample of FIG. 3 0 is the most probable symbol. FIG. 3 illustrates anexample 12 bit symbol set that can be coded using only 9 bits due to thegrouping. Larger grouping lengths (such as 3, 4 or more) may be used tofurther improve the compression efficiency if the probability of thebinary symbols is more biased.

FIG. 4 illustrates an example grouping with length 2 for refinementelements using the same example as illustrated in FIG. 2. FIG. 4illustrates that symbols of the same element type are grouped and codedaccording to the grouping length. Any remaining ungrouped symbols ofthat element type are treated as fractional symbols. Such fractionalsymbols are coded individually. Thus FIG. 4 illustrates grouping therefinement flags into groups 10, 11 and 10 with a fractional symbol 1.FIG. 4 also illustrates grouping the refinement signs into groups 00 and01 with a fractional symbol 0. Note the arithmetic data of therefinement magnitude is not grouped.

Fractional symbols worsen the compression efficiency because thosesymbols are not coded by more efficient VLC tables. In an ideal case,the symbols of one element type in the whole layer are coded at once toachieve the maximum compression efficiency because it minimizes thenumber of non-grouped fractional symbols. However, due to the demand forfine-grained scalability, the coefficients in one layer tend to bedivided into smaller pieces depending on their frequency band. The lowerfrequency band has higher priority. Due to memory bandwidth restrictionseach piece is further divided into smaller coding units based on theirspatial location by smaller rectangular region called codeblock ormacroblock. Therefore, there will be many coding units within one layerand each coding unit will contain few coefficients in actualimplementation of quality scalable video coding technology. In thesecases, there will be many fractional symbols causing compressioninefficiency.

When fine-grained scalability is required, a refinement magnitude ofonly 0 or 1 should be allowed and the number of non-zero refinementcoefficients should decrease. In this case, many coding units containfew symbols (i.e. 0 or 1) for the refinement sign. This creates manyfractional symbols, which reduces compression efficiency.

Therefore, VLC based technique that codes the elements required forrefinement of magnitude more efficiently without losing thefunctionality of image quality scalability would be desirable.

SUMMARY OF THE INVENTION

The present invention is an improved compression efficiency method ofvideo coding with quality scalability. This method extends theinformational elements coded by a VLC technique. This invention includesof the following steps:

Group the first binary element of magnitude refinement process. This isthe refinement flag in the example of FIGS. 2 and 4. This groupingdepends on the probability balance of 0 and 1. Then code the groupedsymbols using the VLC table prepared depending on the probabilitybalance of 0 and 1. Note this step is optional.

Compare the probability balance of the current binary element and thatof the next binary element.

If the probability balance of the current binary element and that ofnext binary element is completely different, then code the fractionalsymbol of the current binary element and the next binary elementindependently. The next binary element is the refinement sign in theexample of FIGS. 2 and 4. An example of a completely differentprobability balance is change from a 0:1 balance of 90%:10% to a 0:1balance of 50%:50%.

If the probability balance of the current binary element and that ofnext binary element is similar but of opposite polarity, then everybinary symbol of the next binary element will be bit-reversed andbundled with the fractional binary symbol of the current element. Anexample of a similar balance of the opposite polarity is a change from a0:1 balance of 90%:10% to a 0:1 balance of 10%:90%. Then the bundled bitsymbols are grouped and coded according to the grouping length and VLCtable used for the current layer.

Otherwise if the probability balance of the current binary element andthat of next binary element is similar and its polarity is the same,then the binary symbols of the next binary element will be bundled withthe fractional binary symbol of the current element. An example ischange from a 0:1 balance of 90%:10% to a 0:1 balance of 88%:12%. Thenthe bundled bit symbols are grouped and coded according to the groupinglength and VLC table used for the current layer.

These steps repeat for each subsequent binary element to be coded in themagnitude refinement process.

Any remaining ungrouped binary symbols are then coded.

Any remaining non-binary elements of this magnitude refinement processare coded using some VLC technology.

This invention decreases the number of fractional binary symbols to becoded. This is advantageous because these fractional binary symbolswould worsen the compression efficiency. This invention is applicablealone or in combination with other schemes for improving video codingefficiency.

BRIEF DESCRIPTION OF THE DRAWINGS

These and other aspects of this invention are illustrated in thedrawings, in which:

FIG. 1 is a flowchart illustrating the conventional VLC refinementsignal coding (prior art);

FIG. 2 is an example of the conventional VLC refinement signal coding(prior art);

FIG. 3 illustrates the grouping of symbols according to the prior art;

FIG. 4 illustrates an example of grouping of symbols for refinementelements according to the prior art; and

FIG. 5 is a flowchart illustrating refinement symbol coding of thisinvention.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

FIG. 5 is a flowchart illustrating method 500 of this invention. Themethod of this invention is based on the two ideas. First, theprobability balance of a binary informational element tends to be biasedespecially for the enhancement layers closer to the base layers. Thus,it is expected that each binary element tends to have the similarprobability balance and grouping distinct element symbols should be lessproblematic. Second, grouping distinct element symbols should reduce thenumber of fractional binary symbols that make VLC coding inefficient.Accordingly, this invention improves compression efficiency by employingsmart grouping of distinct element symbols.

Technique 500 begins with start block 501. Block 502 scans thecoefficients and identifies those to be coded using this process. Block503 divides each refinement coefficient into several informationalelements. These blocks are the as respective blocks 102 and 103 of theprior art technique illustrated in FIG. 1. Test block 504 determines ifthere are any remaining fractional symbols. If there are no remainingfractional symbols (No in test block 504), then block 505 groups thecurrent binary informational element symbols. Block 506 codes thegrouped symbols using a known VLC technique. These are similar to theprior art of respective blocks 104 and 105 illustrated in FIG. 1. Testblock 507 determines if there are remaining binary informationalsymbols. If there are remaining binary informational symbols (Yes attest block 507), then block 508 starts the next binary informationalsymbols. Technique 500 returns to block 504 to start coding this nextbinary informational symbol. These processes are the same as respectiveblocks 107 and 108 illustrated in FIG. 1.

If there were fractional symbols remaining (Yes at test block 504), thentest block 509 determines whether the probability balance for theprevious and current elements are similar. If this probability balancewas not similar (No at test block 509), then block 510 codes theseremaining fractional symbols. In this case no advantage can be achievedby coding these fractional symbols with the following grouped symbols.Thus they are independently coded. Optional block 511 groups the currentbinary element symbol elements. Block 512 codes these grouped symbolsusing a VLC technique. Test block 507 determines if there are remainingbinary informational symbols. If there are remaining binaryinformational symbols (Yes at test block 507), then block 508 starts thenext binary informational symbols.

If this probability balance was similar (Yes at test block 509), thenblock 513 groups these remaining fractional symbols with the currentbinary symbol elements. In this case, it is expected that groupingsymbol elements across the previous and current binary element symbolswould be advantageous. If the probability balances are similar but ofopposite polarity, it is expected that the compression efficiency can bemaximized by matching the more probable symbol of current element withthat of previous element. In this case block 513 bit reverses thecurrent binary element symbols. Block 513 then groups the symbols of thecurrent element with the fractional symbols of previous element. Block514 codes these grouped symbols using a VLC technique. If there areremaining binary informational symbols (Yes at test block 507), thenblock 508 starts the next binary informational symbols.

If there are no remaining binary informational symbols (No at test block507), then block 515 codes any remaining fractional symbols. Block 516codes any remaining non-binary elements using a VLC technique. Technique500 ends at end block 517.

Statistical evidence for the effectiveness of the invented method isdescribed in Tables 1, 2 and 3. Table 1 shows the conventional VLC tablefor 3 bit binary symbols.

TABLE 1 Grouped Symbols Codeword 000 0 001 010 010 011 011 11100 100 110101 11101 110 11110 111 11111Coding with this VLC table will achieve higher compression efficiencythan direct binary flag coding if the probability of 0 is greater than⅔. Table 2 shows the joint probability of both refinement flag andrefinement sign having symbol 0 with the probability greater than ⅔using the SVC (Scalable Video Coding) video coding standard standardizedas an amendment to the H.264/MPEG-4 AVC standard for various images.

TABLE 2 Joint probability Joint probability of “probability of of“probability of Sequence 0” > ⅔ MPS” > ⅔ BusQCIF 0.729 0.733FootballQCIF 0.714 0.723 ForemanQCIF 0.696 0.765 MobileQCIF 0.674 0.786CityQCIF 0.701 0.735 CrewQCIF 0.581 0.738 HarbourQCIF 0.804 0.809SoccerQCIF 0.741 0.764 BusCIF 0.811 0.811 FootballCIF 0.680 0.689ForemanCIF 0.663 0.721 MobileCIF 0.690 0.759 CityCIF 0.767 0.791 CrewCIF0.657 0.784 HarbourCIF 0.736 0.740 SoccerCIF 0.787 0.804 City4CIF 0.7600.765 Crew4CIF 0.615 0.756 Harbour4CIF 0.706 0.754 Soccer4CIF 0.7840.789 Average 0.715 0.761 Minimum 0.581 0.689 Maximum 0.811 0.811The probabilities of 0 for the refinement flag and the refinement signwere measured for each enhancement layer of each coded picture. Theprobability over all the enhancement layers that both refinement flagand refinement sign had probability of 0 or a probability of the moreprobable symbol (MPS) greater than ⅔ was measured. As shown in Table 2,an average of more than 70% of enhancement layers are expected toimprove in compression efficiency employing this invention bundling thebinary symbols of distinct elements. Upon introduction of the bitreversing technique, an average of more than 75% of enhancement layersbenefit in compression efficiency.

Table 3 shows the results of using this invention. This inventionresults in improvement of compression efficiency. Table 3 shows thenumber of coded bits needed to represent refinement symbols with andwithout the invention using the one of the official test conditions ofthe SVC standard.

TABLE 3 SNR Scalability Coded Bits for Bit Test Refinement SymbolsReduction Condition Prior Art Invention (%) BusQCIF 543,672 538,409−0.97% FootballQCIF 1,934,486 1,922,671 −0.61% ForemanQCIF 547,711846,054 −0.30% MobileQCIF 874,995 865,834 −1.05% CityQCIF 674,255669,438 −0.71% CrewQCIF 1,186,048 1,182,662 −0.29% HarbourQCIF 1,401,8941,382,728 −1.37% SoccerQCIF 1,011,857 1,005,117 −0.67% BusCIF 2,019,6152,002,910 −0.83% FootballCIF 7,190,479 7,169,359 −0.29% ForemanCIF1,959,847 1,953,504 −0.32% MobileCIF 3,409,871 1,953,504 −1.34% CityCIF2,532,930 2,513,988 −0.75% CrewCIF 4,730,987 4,743,358 0.26% HarbourCIF6,019,396 5,938,032 −1.35% SoccerCIF 4,207,685 4,181,351 −0.63% City4CIF19,752,531 19,538,184 −1.09% Crew4CIF 22,027,283 22,124,102 0.44%Harbour4CIF 27,226,782 27,087,042 −0.51% Soccer4CIF 18,728,16918,688,922 −0.21% Average −0.63%Table 3 shows the invention does not always result in bit-reduction asin the CrewCIF and Crew4CIF sequences. However, the bit-reduction occursin most sequences. Also, the overall bit-reduction is estimated as 0.63%on average.

There are minor concerns on increasing complexity and worsening errorresilience. However, the increase in complexity increase of theinvention is negligible because the scanning process to detect thecoefficients to be refined is much more complex. The invention will notbe processed over the existing boundaries of spatial location andfrequency band. The error resilience should not be an issue becausesubsequent binary symbols are dependent on the preceding binary symbols.Thus the negative effects of this invention are negligible.

This invention results in reduction of the amount of coded bits forrefinement symbols by 0.65% on average in one of the official testcondition of video coding standardization activities. The negativeeffects of the invention are negligible.

What is claimed is:
 1. A method of transforming a video into a codedcompressed video with improved compression efficiency and qualityscalability including coding refinement symbols video with qualityscalability that contains the binary informational elements, comprisingthe steps of: classifying refinement coefficients into at least onebinary element symbol and any remaining multinary symbols; determining aprobability balance between 1 and 0 for each of a prior binaryinformational element and a current binary informational element; if theprobability balance of the prior binary informational element is similarto the probability balance of the current binary informational elementgrouping any fractional symbols of the prior binary informationalelement with the current binary informational element, and coding saidgrouped symbols using a variable length coding technique; if theprobability balance of the prior binary informational element is notsimilar to the probability balance of the current binary informationalelement coding any fractional symbols of the prior binary informationalelement, grouping symbols of the current binary informational group; andcoding said grouped symbols using a variable length coding technique. 2.The method of claim 1, wherein: said step of determining a probabilitybalance between 1 and 0 for each of a prior binary informational elementand a current binary informational element includes estimates from thepreviously coded symbols.
 3. The method of claim 1, wherein: said stepof determining a probability balance between 1 and 0 for each of a priorbinary informational element and a current binary informational elementincludes computing by pre-scanning refinement coefficients to be coded.4. The method of claim 1, wherein: each of said steps of grouping anyfractional symbols of the prior binary informational element andgrouping symbols of the current binary informational group is unchangedthroughout an enhancement layer.
 5. The method of claim 1, wherein: eachof said steps of grouping any fractional symbols of the prior binaryinformational element and grouping symbols of the current binaryinformational group is configurable in a localized region.
 6. The methodof claim 1, wherein: if the probability balance of the prior binaryinformational element is similar to but opposite the probability balanceof the current binary informational element bit-reversing symbols of thecurrent binary informational group; grouping any fractional symbols ofthe prior binary informational element with the bit reversed currentbinary informational element, and coding said grouped symbols using avariable length coding technique.